How complicated must an optical component be?

We analyze how complicated a linear optical component has to be if it is to perform one of a range of functions. Specifically, we devise an approach to evaluating the number of real parameters that must be specified in the device design or fabrication, based on the singular value decomposition of the linear operator that describes the device. This approach can be used for essentially any linear device, including space-, frequency-, or time-dependent systems, in optics, or in other linear wave problems. We analyze examples including spatial mode converters and various classes of wavelength demultiplexers. We consider limits on the functions that can be performed by simple optical devices, such as thin lenses, mirrors, gratings, modulators, and fixed optical filters, and discuss the potential for greater functionalities using modern nanophotonics.